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Hie Director 

Central Intelligence Agency 



Washington. D. C 20505 


10 AUG 1977 



Ms. Rosemary De Carlo 
Office of Assistant Librarian 
Room 110, Main Building 
Library of Congress 
Washington, D.C. 20540 




Dear Ms. De Carlo: 


Enclosed is my authorization for the 
Library of Congress to retain and utilize 
the recording of my presentation to 
Congressional Interns in the Coolidge 
Auditorium on 13 July 1977. 


I hope that the views I expressed are 
useful to future researchers and I would be 
delighted to respond to any inquiries which 
may be directed to the Library dealing with 
the substance of this recording. 


Yours sincerely, 

/ - 




^ 

STANSFIELD TURNER 


Enclosure 


Distribution: 
Original - 
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Addressee w/encl 
ER w/encl . 
Acting DDCI 
w/ end . 

DC I w/enc. 

OLC Sub j e( 
w / encl . ' 

OLC C hrorfo 
w/o end. 
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PERMISSION TO RECORD IN THE 
LIBRARY OF CONGRESS 


I, STANSFIEED TURNER 

(Name ) ~ 

hereoy grant to the Library of Congress the right to record my 


and to use this 


presentation on 13 July 1977 

' (Date) 

recorded material for reference, research, and for cultural and 


educational purposes. 

I authorize the Library of Congress to provide single 
copy reproductions of it at cost in response to ’requests from 
educational and' cultural institutions. 

This permission is specifically limited to the use' 
descrioed above. I reserve to myself all other rights which I 
possess in this recorded material. 
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OLC 77-2878/1 


MEMORANDUM FOR 

VIA 

FROM 

SUBJECT 


Director of Central Intelligence 

A/DCI/PA 

George L. Cary 
Legislative Counsel 

Library, of, ^Congress. Request for' Tape- - 
Recording of-. DCI Briefing- of- Congressional 
Interns (13 July 1977) 


1. Action Requested : It is , requested •that'‘'y°tr autho-riae-'-'*-*- 
the Library of Congress to retain,: as a public record,.. .at ape- 
recording of the-- talk before- the Congressional* Interns on - 

13 July -19 77. 

2. Background : On 13 July 1977, you appeared before a 
group of 600 Congressional Interns at the Library of Congress 
Collidge Auditorium to discuss "The CIA." The Library of 
Congress has informed us, after the fact, that the -entire 
session- .was tape-recorded as is their customary practice for 
all such Congressional Intern briefings in the Coolidge 

Auditor ium . The L-ib-ra-ry'-ef - Congress- -is- now-.- s eeking your 

approval to place* that recording into the public domain . 

3. Staff Position : We have: reviewed the tapes and 

re commend-, that ..they be . released to the. .Library of Congress-, as 
the public record. The copies which -were given to us for 
r e v i e» « wd 1 1 be ■■ f orwa rded t o A/ DCI / PA., for re ten t ion . 

4. Recommendation : It is requested thatybu' sign -the 

attached authorization form and covering note for transmittal 
to the Library of Congress. I . 


/JGeor.gb L. Car/- 


Attachment : 
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The Honorable Les As pin 
House of Representatives 
Washington, D. C. 20515 

Dear Les: 


Thanks for providing a copy of your analysis of Soviet 
Shipbuilding Forecasts. It has, of course, been provided to the 
appropriate people in the CIA. I trust that you have provided the 
study to the Department of Defense which was the source of much of 
your basic data. 

It is clear--in retrospect— that many of the estimates you 
cite have proven to be considerably inaccurate. Where the Intelligence 
Community is involved in making these estimates and projections we 
shall continue our efforts to improve their quality and better describe 
the basis for our projections and our confidence in them. 

There is, however, no high likelihood that the accuracy of 
Defense or CIA projections will be constantly improving over time. 

This is a dynamic affair and as we improve our collection capability 
on the one hand the Soviets take actions to deny information on the 
other. This is not to say that we are complacent about poor quality, 
however. We should try to refine our estimating capability based on 
lessons learned and better data. 


I appreciate your interest in intelligence collection and 
analysis. 





ocutt! 

« 
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THE DIRECTOR OF CENTRAL INTELLIGENCE 


WASHINGTON, D. C. 20505 



National Intelligence! Officers 


DCI/NIO 1932-77 
4 August 1977 


MEMORANDUM FOR: 

Director of Central Intelligence 

VIA : 

George L. Cary 

Legislative Counsel 

FROM : 

Robert R. Bowie 

Deputy to the DC I for National 

Intel! igence 

SUBJECT : 

Aspin Study of Soviet Shipbuilding 

Forecasts 

REFERENCE : 

Letter from Congressman Aspin to DCI, dated 
27 July 1977 transmitting subject study 


Re 9 ues - ^-‘ Representative Aspin forwarded by personal 

asked that thL C ?rn S ° f A StUdy ° f ? oviet Shipbuilding Forecasts and 
asked that, they be passed to appropriate people in CIA. No response 

was requested but a brief acknowledgement is considered in order and 
a proposed response is attached. na 


■69 thrn., 7 ^73 The ^ tudy ^ ln 9 intelligence projections from 

arhifS 3 analyzes the quality of the estimates compared with 
fmrill lev ® ls in such a manner as to allow the study to be 

cSmbataitr/^innff wfV projec Ji°! 1s; category-major surface 
combatants (>1000 tons), large amphibious ships, SSNs, SS— for 

deliveries in 1970 through 1975; and by numbers of years into the 

Fininv Pr ?f Ctl ° nS madS ^ e - g ” 2- or 3-year projections). 

s tudy c‘oSdes: ** » r0 J ectl ' 0 " s fo straight-line projections. 


, a l regard to time (value or range), the estimate 

t?L d pr Ct T at 1 29 ?- rC ? n ? of the time ’ overestimated 57 percent of the 
time and underes timated the remaining 14 percent. (Data in the 

estimates were presented as the 75 percent confidence level.) 

Aral «, A b » the diesel submarine category had a balance of "over" 

f u n d ^ f o r r d i p ^ p l S Soviet P r °duction and the US was not seeking 
tundj for diesel submarine procurement duri ng this time. 
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c. Without regard to time, in terms of the calculated 
percentage inaccuracy the "overestimates" were from 2.5 to 4.5 times 
greater than the "underestimates." 

d. The 1-, 2- and 3-year projections do not improve in 
quality over time but rather their quality deteriorates. 

e. Though not advocated, that a straight-line projection 
would have been four times as accurate as those made for all categories 
but the diesel submarine projection which was more accurate than 
straight-line projection. 

f. The proclamation that we underestimate Soviet programs 
(Keegan, Team B, Wohlstetter) is not supported by the facts. 

g. If we continue to overestimate Soviet shipbuilding, 
we overestimate Soviet naval capability. 

3. Staff Position . The NIO (Conventional Forces) with the support 
of the Naval Branch of OSR has reviewed the study. The study has some 
deficiencies and there are some reasons for the poor quality of the 
estimates. These are outlined in Attachment A. 


Though Representative Aspin refers to DIA as the source of 
data for the years '69 and '70, the estimates were "national" vice 
"defense" estimates and responsibility for error is not exclusively 
that of the Defense Department. 

4. Recommendation. A reply be made to Representative Aspin which: 

--acknowledges the study; 

— expresses a commitment to improve quality of Community 
products but recognizes some realistic impediments; 

— but does not evaluate the study or excuse the error. 

A proposed reply (Attachment B) is attached for your consideration. 


Kooert k. bowie 


25X1 


Attachments: 

(A) -Comment on the Aspin study of Soviet Shipbuilding 

Forecasts 

(B) -Draft reply to Representative Aspin 
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Distribution: NIO 1932-77 
0-Adse w/atts 
1 - Mr, Cary, LC w/atts 
1 - D/DCI/NI w/atts 
1 - ER w/atts 

1 - NIO Reg w/atts 

2 - NIO/CF w/atts 
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Comments on the Aspin 
Study of Soviet Shipbuilding Forecasts 


1. The study refutes statements by Team B, Wohlstetter and 
Keegan that the US regularly underestimates Soviet forces— referring 
to Strategic Forces in the main — by analyzing and proving that our 
projection of non-strategic shipbuilding has been consistently 
overestimated. 

. 2. For the years 1969 and 1970, which are used by Representative 
Aspin, the Intelligence Projection for Planning was a National estimate 
vice a Defense estimate, therefore, responsibility for error is not 
exclusively that of the Defense Department. The errors were, however, 
smaller in those years and they tend to create a base from which 
estimates have somewhat deteriorated. This is not a point made or 
even implied by Rep. Aspin; it just happens to be so. 

3. The implication in the study that projections in the diesel 
submarine category were most accurate because Defense did not need to 
exaggerate this category since no funds were sought for diesel sub- 
marines possesses faulty logic. The Soviet diesel submarine would 
not justify US diesel submarines but does constitute a threat which 
would affect US requirements for ASW weapons, sensors and systems. 

Though not as important as the SSN threat, this diesel threat is, 
nevertheless, important. 

4. Some computation methods are inherently biased. 

a. The method of computing error in an estimate as a percent 
of actual will always show overestimates as larger than under- 
estimates for a fixed value of error. 

Example : Assume an estimate of ten units and an error 
of two. The overestimate of two produces an inaccuracy 
percent of 2/8 = 25%, the underestimate of two produces 
an inaccuracy percent of 2/12 = 16.6%. 

b. In the evaluation of one, two and three-year projections, the 
ratio of projected to actual is unnecessarily inverted to "weigh 
each type of error equally." This will always produce a larger 
ratio for overestimates than underestimates for a given value of 
error and the larger the error, the larger the bias. 
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E_ xamp1 e : For an actual production of ten units and an 

a ZioofZ I n 2 ‘- e T rest1ma ‘* else Reduces 

a ratio of lnl 2 -~ r 83 the V nde r estlma tes case produces 

a ratio of 8/10 - .8. If the projection error was 4, the 

ratios would be .71 and .6, respectively. 

c. Where a range of values at the .75 confidence level 
a^ r thp X n!ff Sed 1n k h + estlmate » to describe the value of the error 

c.A does ssrjoiM ivtirri n: ot been verified b — 

eva,ufied^:1^2?: MS 

of whfch were del?^a e te ma ™ey Sbipbu11d1ng "»*««*.. none 

fl ^P ec tatation that the new classes of submarines, the 

Alfa and Fap, would follow previous classes. In fact these 
programs terminated with a combined total of seven SSNs; 

b. Expectation that SSBN building would taper off and 
capacity would shift to SSNs; y P and 

not lay nearly 3 idfe;^ 3 relatively modern shipyard-GORKY— would 

d. The rising tide of the Red Navy with their CNO on Timp 
magazine s cover, his extensive writing and the impressioniT? 
the increased Soviet perception of a Navy's value and the 
analysts' perceptions of its likely continued rapid growth; 

e. Failure of analysts to appreciate that as the Soviet shins 

nu n mbers Se dei?ve?ed n ! ge there W ° Uld be 3 comparab,e 1" 

1 imitations* Ure t0 project Soviet manpower and overhaul 
unpl easant"surpr|_ses. nature of P la "" ers *> a™™ 

$' ’ >!J f : 
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Congress o i tfje feiteb States! 
use of iEepcetfentatfofcS 

SSaa&toBtow, 38.C. 20515 


July 27/ 1977 

Admiral Stansfield Turner 
Director 

Central Intelligence Agency 
Washington, D.C. 

Dear Stan: 


EwcuSya F«.jv,itr*- | 



00400(22-8 


PDl 

/jLrCi Jtt't ^ *• " 

i /w ? - <- ys*- t 1 

}£ : > , yve /Xj /&>>*■'• 


Enclosed are several copies of a 
study I did recently on ship- 

building forecasts. 

Please pass it Jon to those at the 
CIA who follow this [sort of thin®. 


MU' 


Lei Aspin 

Member of Congress 


IAAc 

Enc. 


i "i" »• 

H \ 


K\ 


m 


7 

1 A — * 
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SOVIET S'!IP0( ! ILDI 16 RECASTS: 

HO*'* INTELLIGENT IS U.S. INTELLIGENCE? 


A Study by 
?ep. Les Asp in 


July 12 , 1277 
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The U. S * intelligence services ere under attack not only for 
assassination plots and assorted other dirty tricks but also for their 
estimates of what the Soviets are up to. 

The most recent example is the "Team E " criticisms of the CIA's 
interpretation of what motivates the Russians. 

Before that, Albert Wohlstetter argued in a series of articles in 
Foreign Policy that past estimates of Soviet strategic forces had con- 
sistently underestimated the pace of Russian expansion. 

Then there is Maj. Gen, George I. Keegan, who, since his retirement 
as chief of Air Force intelligence in January, has fired innumerable 
salvoes at our intelligence output. "The United States, with rather 
remarkable consistency, has underestimated the Soviets in almost every 
major field of science, weaponry and force development, " he recently 
wrote in the Christian Science Monitor. 

This is not just a bureaucratic spat. The estimates of what the 
Soviet military is doing today and the projections of what it will do 
tomorrow are key points that the Congress and the public look at during 
the annual budget debate. 

This is certainly true in the annual debate over the Navy*s ship- 
building budget request. The Administration is requesting, and Congress 
is appropriating, funds for vessels to be built in future years based 
on what intelligence says the Russian Navy will have in future years. 
These estimates come from the Defense Intelligence Agency (DIA) and are 
a product of the military intelligence system. 

There is little doubt as to their importance. The Defense Depart- 
ment has taken the position that a "basic" factor in determining the 
requirement for the sise and composition of the U.S. Navy is "the threat 
expressed as intelligence estimates of future Soviet maritime capabil- 
ities ." 1 Similarly, the chief of naval operations, Adm. James Holloway, 
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has testified that the “accuracy of the intelligence estimates of future 

Soviet maritime capabilities” is one of the “three basic factors" 
determining future force requirements, J 

Therefore it is important to chec3: the quality of our projections. 
This study gathers the estimates of Soviet ship output made in past 
years and compares them with actual Russian production v/hen those years 
rolled around. 


METHODOLOGY 

This analysis is based on official projections made in the five 
3 

years 1969-1973 for the number of Soviet warships to be delivered in 


the years 

1970 through 1975, as shown in Table 

Table I. Format of Shipbuilding 

I. 

Estimates 



Year- Pro j 

action 




Jfc'Cl.3 

Year of 

Delivery 




1970 1971 .1972 

1973 

1.974 

1975 

1969 

x-y x-y x-y 

x-y 

x-y 

x-y 

1970 

x-y x-y 

x-y 

x-y 

x-y 

1971 

x-y 

x-y 

x-y 

x-y 

1972 


x-y 

x-y 

x-y 

1973 



x-y 

x-y 

Actual 

Productions z z z 

E 

s 

s 


The DIA 1 s estimates were usually expressed as a range (a low of x 
to a high of y) , although sometimes single point predictions were made. 
Letters are used in the table for illustrative purposes since actual 
forecasts of Soviet production remain classified. 

The projections examined were for each, of four categories of 
general purpose ships % major surface combatants of 1,000 tons or 
more^y large amphibious ships? nuclear-powered attack submarines? and 
diesel-powered attack submarines. As Table I demonstrates, there were 
20 estimates in each category. With four categories, we are dealing 
with 80 estimates. 


Approved For Release 2004/03/17 : CIA-RDP80M00165A001800040022-8 




- 3 - 

Approved Fdrkelease 2004/03/17 : CIA-RDP80M00W5A001 800040022-8 
X. ACCURACY OF ESTIMATES 

Table IX shows whether the actual production (2 in Table I) fell 
above , below or within the range (x - y ) of each of the 80 individual 
projections . 

Treble II- Accuracy of Projections of Soviet Output 


Ship 

Cateaorv 

Overestimates 
(Output lower than 
estimated range) 

True Estimates 
(Output within 
estimated range) 

Unde re st imate s 
(Output higher than 
estimated ranee) 

Major Surface 

Warships 

11 

7 

2 

Amphibious 

Ships 

12 

5 

3 

Nuclear Attack 

Submarines 

19 

0 

1 

Diesel Attack 

Submarines 

4 

11 


Total 

46 

23 

11 

(as % of all) 

(57%) 

(29%) 

(14%) 

Table II 

shows that the DIA overestimated actual 

Soviet production 

more than half 

the time and in three 

of the four ship 

categories; only 


diesel sub production fell within the range of predictions in the 
majority of instances. And half of all the underestimates are in the 
diesel sub category. 

The table shows that only 29% of the forecasts turned out to be 
accurate. This record is particularly remarkable in light of DIA claims 
of a 75% probability that the actual production figure will fall within 
the range predicted. 

Of course, we cannot expect our intelligence services to forecast 
Soviet output perfectly. Some errors are unavoidable. But the key 
point illustrated by Table II is the direction of the inaccuracy* the 
errors were overestimates four times as often as they were underestimates. 
Again, only in the diesel attack submarine category do we find that the 
errors fall relatively equally into overestimates and underestimates. 
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The United States has not produced any diesel subs for years and 

clid not produce any during the tiitie period of this analysis. It is 

interesting and perhaps coincidental that relative accuracy and an even 

balance of under- and overestimates were achieved only in that category 

for which the Defense Department was not trying to convince Congress of 

the need for funds. 


II. MEASURING THE INACCURACY 

The analysis shown in Table II has its limits. A prediction is an 
error even if the prediction was off by only one ship, e.g., if the 
forecast were for 8 to 10 ships and the actual production was 11. 

A fair assessment requires that we measure the degree of inaccuracy. 

To do this, the actual ship output has been compared with the midpoint 
of the predicted range (or the point estimate where no range was pro- 
vided) in all of those cases where output fell outside the range predicted. 

The degree of inaccuracy was then calculated by dividing total errors 
by the total actual output for each category. If the DIA predicted 6 
ship deliveries in a given year and only 4 were delivered, the error 
is 2. The degree of inaccuracy (error divided by actual output) is 2 
divided by 4 or 50%. Similarly, if the DIA had underestimated output 
by 2 ships when actual production was 4, the error is 50%. 

Table III shows the error rate associated with each type of 
inaccurate prediction (the 46 over- and 11 underestimates in Table II) 
made by the DIA. 


Ta ble III. Degree of Error in Projections of S oviet Output 


Ship 

Category 

JL 

Overestimates 
Inaccuracy 
(error as % of 
actual output) 

# 

Underestimates 
Inaccuracy 
(error as % of 
actual cutout) 

Major Surface 
Warships 

11 

64% 

2 

14% 

Amphibious 

Ships 

12 

380% 

3 

100% 

Nuclear Attack 
Submarines 

19 

150% 

1 

33% 

Diesel Attack 
Submarines 

4 

163% 

5 

71% 

Overall: 

46 

108% 

11 

40% 
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Table III shows that the errors in the DIA estimates are substantial. 

In the case of overestimates, the midpoint of the average prediction 
was 108% higher than actual output. In the case of underestimates, the 
midpoint of the average prediction was 40% lower than actual output. 

In every category, the error associated with the overestimates was 
substantially higher than thajt associated with th© underestimates. 

The results of Table II and III can be summarised by considering 
a hypothetical case. If, for a given year, the DIA predicted that 
11-14 ships of a given type would be built? 

— 29% of the time actual Soviet production would have been 
between 11 and 14 ships? 

— 14% of the time the Soviets would have built more than 11 to 14 
ships, and on the average would have built seven more or 21 ships? 

-- 57% of the time the Soviets would have built fewer than 11 to 
14 ships and on average would have built five fewer, or 6 ships. 

Forecasts of this level of accuracy are not very useful. 

III. HAS LEARNING OCCURRED? 

Analy 2 ing the errors by ship category is one way of looking at the 
predictions. Another is to consider the accuracy of the estimates over 
time. A natural question is whether there has been any improvement over 
the years. Does the DIA learn from past mistakes? 

In order to find out, projections made in each year have been 
compared. However, those made in early years like 1969 cannot be com- 
pared in toto with those made later, such as in 1973, because in earlier 
years estimates extended several years into the future. These are almost 
surely more speculative and one should not expect them to be as accurate. 

To adjust for this problem the projections have been grouped by 
the number of years into the future for which they were made. Thus, 
projections for one year into the future made from 1969 to 1973 for each 
class of ships were plotted out* Then, projections for two years into 

the future were compared, and so forth. 

The results are shown in Figure 1, which plots the ratio of 
the predicted deliveries to the actual number of deliveries. (Mien a 
range of deliveries was predicted, the midpoint of the range was used. 

For overestimates, predicted deliveries were the numerator? for Under- 
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estimates, they were the denominator. This weights each type of 
error equally.) 

The Isl ratio is obviously a perfect estimate. If there had been 
learning, the ratio should move closer to this standard over tine. 

But clearly, this has not happened. Indeed, the estimates are more 
inaccurate in 1973 than in 1969. 
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IV. IMPROVING TE.2 ESTIMATES 

Could the estimates be improved? The comparative quality of the 
diesel submarine projections indicates they could. But are we not 
perhaps demanding more than the state of the art can produce? 

The Defense Department clothes its intelligence procedures in an 
aura of mystery. This prompts us to imagine all sorts of sophisticated 
analyses being used to come up with these ship production estimates. 

If the analyses aren't accurate, do we not then need, an even more 
sophisticated system? an even more complex computer? even more intricate 
software? 

For the sake of comparison, let us use not a more sophisticated 
method of projection, but the crudest method possible — straight- lining. 
Simply assume that in the future the Russians produce the same number 
of ships they are already producing. 

Table IV shows that if the DIA had used this method, thereby re- 
ducing the number of man-hours needed to produce its projections by 
about 99%, those projections would have h&en consistently more accurate. 


Table IV. 


Ship 

Catacfory 


Comparison of DIA with tStraicht-Line Estimates _ 
Straight-Line Straight-Line Straight-Line 
More Accurate As Accurate Less Accurate 
Than PIA As DIA Than PIA 


Major Surface 

Warships 11 

Amphibious 

Ships S 

Nuclear Attack 

Submarines 19 


3 

8 

1 


6 


4 

0 


Diesel Attack 
Submarines 


5 


2 


13 


Total 

(as % of all) 


43 14 

(54%) (18%) 


23 

(29%) 


As indicated in Table IV, straight-lining would produce more 
accurate estimates twice as often as it would less accurate ones. 

In fact, the only category where straight- lining failed to give better 
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results was in forecasting diesel sub production, th© on© category which 
was not a problem in the first place* Other than the diesel sub category 
straight- lining produces more accurate results almost four times as 
often as it produces less accurate ones. 

This is not to suggest that we should adopt straight- lining and 
cease any analysis of Soviet ship programs* But it does suggest that 
we ought to find an entirely new framework in which to make such analyses 

CONCLUSIONS 

Two broad conclusions emerge firom this analysis; 

1) the loudness and frequency of proclamations that we consistently 
underestimate the Soviets "in almost every major field of science, 
weaponry and force development" is unsupported by the facts — shrillness 
should not be confused for evidence? 

2) official estimates of Soviet ship production have been grossly 
inaccurate, misleading both the executive branch and Congress about the 
extent of the threat that must be countered through our shipbuilding 
program* 

The first conclusion — that the volume of an argument does not 
necessarily equal the volume of evidence — is self-evident. However, 
no one should generalize from this analysis that other intelligence 
projections are inherently overestimated. We must not generalise from 
the particular about overestimates any more than we accept Gen. Keegan* s 
generalization about consistent underestimates. 

As for the second conclusion — that the DIA estimates have been 
misleading — one does not need to interview congressmen to see if they 
have been swayed by inaccurate data; one need only go to the Navy which 
has pointed to the importance of projections of Soviet shipbuilding. 
Future ship deliveries are supposed to be a driving factor behind the 
growth of the Soviet naval threat. But if projections of the number of 
new vessels continue to be too high, Soviet naval capabilities will 
certainly grow more slowly than now anticipated. 
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^Defense Department response to prepared questions from Sen. John 
Stennis, Fiscal Year 1978 Authorisation , Hearings before the Senate 
Armed Services Committee, 95 Cong. 1 Sess. (1977), pt. 2* p. 917* 

^ Military Posture , Hearings before the House Armed Services 
Seapower Subcommittee, Mo. 95-4, 95 Cong. 1 Sess. (1977), pt. 4, p. 906, 

Estimates made in 1976, 1975 and 1974 are excluded because; no 
data for a year following 1976 are available? none were available for 
years following 1975 when this analysis began? and no official estimate 
of Soviet production was made in 1974. 

^This category combines major surface combatants of 1,000 to 3,000 
tons and of 3,000 or more tons. The combined pattern is entirely 
consistent with the experience for these individual groupings. 
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